Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🥁 Prelude #29

Merged
merged 159 commits into from
Apr 9, 2025
Merged

🥁 Prelude #29

merged 159 commits into from
Apr 9, 2025

Conversation

MitchellShiell
Copy link
Contributor

@MitchellShiell MitchellShiell commented Jan 31, 2025 •
edited by justincorrigible
Loading

Prelude is a tool that enables teams to incrementally build their data platform. By breaking down data portal development into phased steps, teams can systematically verify requirements and user workflows while minimizing technical overhead.

Development Phases

Development progresses through four distinct phases, each building upon the
previous phase's foundation while introducing new capabilities.

Phase Description Software Components Status
PhaseOne: Data Exploration & Theming Display your tabular data in a themable portal with our front-end and back-end search components. CSV-processor, Elasticsearch, Arranger, Stage 🟢 Working
PhaseTwo: Tabular Data Management & Validation Implementation of tabular data submission, storage and validation. All the above with Lyric, LyricDb (Postgres), Lectern and LecternDb (MongoDb) added 🟢 Working
PhaseThree: File Data & Metadata Management Implement back-end file management. All the above with Song, Score, SongDb (Postgres) and Object Storage (Minio) 🟢 Working
PhaseFour: Identity and Access management Configure Keycloak to authenticate users and authorize what they have access too. Empahsis on data access control planning and Keycloak configuration ⚪ For Next Release

Documentation

Detailed documentation can be found in multiple locations:

  • The /docs folder at the root of this repository and within the docker-compose.yml
  • README files within each root directory containing information on the folder's
    purpose and usage
  • Frontend documentation available after deployment at
    http://localhost:3000/documentation

Next Steps

  • Update Conductor Repo to Prelude
    • Copy Main branch to a branch called quickstart
    • Update all documentation references accordingly
    • Merge Prelude to Main

@justincorrigible Submodule documentation PR Updates

@justincorrigible Score needs to be merged and is blocked by jenkins

Note

Prelude will eventually merge with the quickstart as an all in one however this will require the keycloak update as well as significant updates to the current guides on docs.overture.bio. Due to this it will be taken on following phase 4 development, for now to avoid confusion we will replace any mention of conductor with quickstart and update this repo name to prelude.

Note

Eventually the documentation within the docs folder here will be published to docs.overture.bio similar to how every other repo works. This is pending updates that will align admonition syntax and title logic.

  • Windows Run through (WSL2)

    • Phase0
    • Phase1
    • Phase2
    • Phase3

  • Mac/Linux Run through

    • Phase0
    • Phase1
    • Phase2
    • Phase3

  • Windows Updates

    • Symbolic links not cloning
    • Stage build step freezing

MitchellShiell and others added 30 commits December 22, 2024 11:33
@MitchellShiell
Proof of concept development kit
9e908ef
@MitchellShiell
Proof of concept development kit mock data
6e88795
@MitchellShiell
Basic front-end complete, need to add documentation
0b616e4
@MitchellShiell
updated readme and documentation
4aa07eb
@MitchellShiell
removed es volumes
289fd82
@MitchellShiell
readme update
0f59309
@MitchellShiell
fixed typo
db8db90
@MitchellShiell
basic setup
bc4a7c4
@MitchellShiell
basic setup with keycloak files
4f63ea5
@MitchellShiell
pre-seeded keycloakDb
f126291
@MitchellShiell
pre-seeded songDb
2029fec
@MitchellShiell
updated with phaseOne and phaseTwo docker composes
cc87823
@MitchellShiell
updated phaseOne and phaseTwo folders
87955da
@MitchellShiell
stage in root
9129c6e
updated stage folder
2869b8d
functional song swagger component
5ae9b57
functional documentation component
8470f47
functional Auth with keycloak
74f2c86
minor update
9ff4f55
updated gitignore
6dfc04e
moved stage folder, updated docker with build context
0867fa8
temporarily removed persistentStorage folder
d0f3ae8
adding persistent storage back
c939004
documentation template, CORS swagger fix, supplementary files included
bb78fd5
stage downloads fixed and tested
9f07d9e
removed nginx configs
dc62a2b
added architecture diagrams
110f496
removed Ego logic
610e596
reset local databases
ba0d04a
phaseOne fix + stage image seperation
9e8a06e
This was referenced Apr 8, 2025
Merged
Merged
This was referenced Apr 9, 2025
Closed
Merged
Closed
Merged
Merged
Merged
Merged
@MitchellShiell MitchellShiell marked this pull request as ready for review April 9, 2025 20:14
@MitchellShiell MitchellShiell merged commit 9236a32 into main Apr 9, 2025
@MitchellShiell MitchellShiell deleted the prelude branch April 9, 2025 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@MitchellShiell MitchellShiell

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Initiative: 🥁 Prelude
3 participants
@MitchellShiell @joneubank @justincorrigible